Papers by Beatrix Miranda Ginn Nielsen
Prediction Hubs are Context-Informed Frequent Tokens in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Hubness is a tendency for a few points to be among the nearest neighbours of a disproportionate number of other points. |
| Approach: | They show that only large-scale representation comparisons are not characterized by hubness . they show that hubs are the result of context-modulated frequent tokens . |
| Outcome: | The results show that the comparison between context and unembedding vectors does not result in hubness . the findings suggest that hubness is not a negative property that needs to be mitigated when LLMs are being used for next token prediction. |